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ABSTRACT 

In this paper we present preliminary results on a study on the effect of instructional method (observational learning and 
learning by doing) and reflection (yes or no) on academic text quality and self-efficacy beliefs. 56 undergraduate students 
were assigned to either an observational learning or learning-by-doing condition, with or without reflection. The 
participants were tested on academic text quality, self-efficacy beliefs and satisfaction with the instructional method. Our 
preliminary results suggest that there is no main effect of instructional method on academic text quality and self-efficacy 
beliefs. Observational learning and learning by doing seem equally effective for learning how to write a large and 
complex writing assignment, even though participants who learned by doing were more satisfied with the method than 
participants who learned by observing models. We found no effect of reflection. However, there appears to be an 
interaction between instructional method and reflection: in observational learning reflection seems to have a negative 
effect on academic text quality while in learning by doing it appears to have a positive effect. We will repeat the 
experiment in September 2016. 
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1. INTRODUCTION 

Writing a coherent and effective text is a complex and demanding task. A writer has to plan ideas, translate 
these ideas into sentences, and type the sentences on a word processor while monitoring these activities at the 
same time ( Kellogg, 2008). This implies that writers have to carry out different processes, and pay attention 
to many textual characteristics simultaneously and therefore may lose track of their own thoughts during the 
process. This applies especially to those who are learning to write: the learner becomes so closely involved in 
the writing process that hardly any cognitive energy is left for learning from that process. A method that 
allows for a distinction between writing and learning to write is observational learning (Braaksma, et al., 
2004). Observational learning is the process of learning a new task by watching and/or listening how 
someone else performs this task. Learners should be able to infer others’ intentions from action observation, 
process others’ action outcomes and combine these sources of information in order to select behaviors 
leading to desired outcomes later on (e.g. Monfardini, et al., 2013; Rak, et al., 2013). Various studies have 
shown the effectiveness of observational learning in the writing domain (e.g. De Wachter, et al., 2015; 
Rijlaarsdam et al., 2008; Raedts, et al., 2007). Braaksma, et al. (2004) suggest that the effectiveness of 
observational learning in learning to write is a result from the observers’ strong engagement in metacognitive 
activities. Observers internalize, apply, and develop criteria for effective writing by observing the models’ 
writing, identifying and conceptualizing the writing strategies, evaluating the performance of the models and 
reflecting on the observed performances. 

In the current study we want to investigate whether observational learning would be a suitable method for 
university students to learn how to write an academic text. Only a handful of studies, all by Raedts and 
colleagues, have asked this question (e.g., Raedts, et al., 2007). These studies investigated the effects of 
observational learning with undergraduate students on self-efficacy beliefs and text quality by comparing 
observational learning with learning without a model (learning by doing). The studies showed that 
observational learning helped students to bring their self-efficacy beliefs regarding a new writing task to an 
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accurate level. With regards to text quality, the studies showed that students in the observational learning 
condition outperformed those in the learning-by-doing condition: they linked the source material more often, 
and wrote better organized literature reviews compared to the students in the learning -by-doing condition 
(Raedts, et al., 2007). In Raedts, et al. (2007) and most other studies on observational learning, learning 
through modeling, is compared with learning by doing. In learning by doing, learners put their own 
knowledge and beliefs on how a certain task should be done into practice, without observing someone else 
performing the task (Bandura, 1977). The main difference between observational learning and learning by 
doing is the absence of a model in the latter. However, closer inspection of the methods suggests that there 
might be a confounding factor; reflection. In observational learning, reflection is often encouraged by asking 
participants questions about the model’s performance. In learning by doing, reflection is not an intrinsic part 
of the process. Adding reflection to learning-by-doing could therefore make these two types of learning more 
comparable. Could reflection account for the differences found between observational learning and learning 
by doing? Reflection within observational learning in the writing domain has been studied by Braaksma, et 
al. (2002). In their experiment participants observed two models performing a writing task. One group started 
with the writing task immediately after the observation, whereas two other groups reflected upon the models. 
Both reflection groups outperformed the no reflection group on the writing tasks. However, this was not 
compared to learning-by-doing. Additionally, the influence of reflection on learning in general has been 
studied. Reflection is found to be beneficial under certain circumstances (see i.e. Bulman and Schutz, 2013). 
Asking learners to monitor or record their performances during their learning process enables them to detect, 
attribute and correct their errors (Zimmerman and Kitsantas, 2002). 

The common belief in educational research seems to be that both observational learning and reflection 
have a positive effect on learning. However, how reflection interacts with observational learning and learning 
by doing is not clear yet. This raises the question whether reflection causes the differences between 
observational learning and learning by doing. In the current study we investigate the effect of instructional 
method (observational learning, learning by doing) and reflection on academic text quality and writing 
self-efficacy beliefs. We hypothesize that observational learning positively influences academic text quality 
and self-efficacy beliefs compared to learning by doing. Secondly, we expect reflection to lead to higher 
academic text quality and self-efficacy beliefs than no reflection. Thirdly, we hypothesize that reflection 
reduces the effect of instructional method on academic text quality and self-efficacy. If observational 
learning proves to be at least as effective as learning by doing, educators could consider this method as part 
of a flipped classroom environment in which students gather information largely outside of class (Berrett, 
2012 ). 


2. METHOD 

2.1 Design 

A 2 x 2 design was used in this experiment, with instructional method (observational learning, learning by 
doing) and reflection (yes or no) as the independent variables. In the posttest the effect of instructional 
method and reflection on academic writing performance, self-efficacy and satisfaction were measured. In this 
paper we only report preliminary results. Due to the small sample size, we will repeat the experiment in 
September 2016. 

2.2 Participants 

The participants were recruited from an obligatory introductory course on academic writing for 
undergraduate students at a Dutch University. The sample consisted of 56 participants (male = 13, female = 
43). The average age was 18,6 years (SD = 1.41). The participants were divided into four tutorial groups. 
Each group was assigned to one of four conditions: observational learning with reflection (n = 9), 
observational learning without reflection (n = 10), learning by doing with reflection (n = 19) and learning by 
doing without reflection (n = 18). 
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2.3 Materials and Procedures 


The interventions took place during the first two tutorials of the academic writing course. All four conditions 
consisted of two interventions. In the observational learning conditions five videos were shown: two during 
the first intervention and three in the second intervention. The participants received a handout that contained 
the introduction to the videos and three index cards that were used in the videos by the actors. The index 
cards contained a summary of a scientific article, which consisted of the full reference of the article; the 
research question, the type of research and data; and a summary of the most important findings of the study. 
The first and third index card represented a similar viewpoint. The second index card contained an opposing 
view point. 

In each video the participants saw two peer models writing an introduction to an academic paper based on 
these three index cards. Each video focused on a certain aspect of writing an introduction to an academic 
research paper. See Table 1 for the content of the videos. 

Table 1. Content of the Videos and Exercises 


Intervention (duration) Video / Exercise 

1 (45 minutes) 1 

2 

2 (45 minutes) 3 

4 

5 


Content 

Reading, selecting, organizing and paraphrasing the 

information on the index cards 

Planning the content and main structure 

Writing the body of the introduction 

Adding an opening and scientific / social relevance 

Revising the text at text, sentence and word level 


The models in the videos were student actors who had received a script for each exercise and had been 
instructed to think aloud during the exercise. One of the models used effective writing strategies to complete 
the assignments (strong model), the other model used counterproductive strategies (weak model). Each 
fragment contained a recording of the model working on the computer, the model’s voice and the computer 
screen the model was working on. In the observational learning with reflection condition participants were 
asked after each video to reflect on what they had seen. They answered questions as: which differences 
between the two writers did you observe and who do you think was the better writer and why? In the 
observational learning without reflection condition the instructor started the next video immediately after the 
previous one. The learning-by-doing condition took place in a computer room. Each participant had access 
to a computer. The instructor told the participants that they would be writing an introduction to an academic 
paper based on three index cards by executing five pre- structured exercises: two during intervention 1 and 
three during intervention 2. The exercises the participants executed were the same as the tasks the student 
actors performed in the videos. The participants received a handout with the introduction to the exercises and 
the three index cards they had to use during the exercises. These index cards were identical to the ones the 
student actors in the videos used. The five exercises were displayed one by one on a screen. In the learning- 
by-doing condition with reflection participants were asked to reflect on their activities after each exercise. 
They answered questions such as: How did you handle the last exercise? What would you do differently next 
time? In the learning-by-doing condition without reflection each exercise was followed immediately by the 
next one. 


2.4 Measures 

To measure the participants’ academic writing performance after the interventions the first author scored the 
introduction section of the first paper the participants had to write for the academic writing course. The 
participants were provided with three index cards similar to the ones used in the interventions. The studies on 
index card 1 and 3 showed similarities in their results, while the study on index card 2 displayed an opposing 
viewpoint. The participants were instructed to write an opening for their introduction, to include all three 
index cards in the body of the introduction and to make sure that the introduction would lead to the research 
question and hypotheses. The texts were scored on ( 1 ) opening paragraph (2) similarity between index card 1 
and index card 3 (3) contradiction between index card 1/3 and index card 2 (4) mentioning of scientific 
relevance (5) mentioning of social relevance and (6) structure in general. For each item two or three points 
could be appointed which resulted in a possible maximal score of fifteen points. Self-efficacy was measured 
with ten items of which the participants had to indicate on a scale from 0 (not confident at all) to 100 (very 
confident) how confident they felt that they were able to perform a certain task. An example of an item is: “I 
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am able to paraphrase information from index cards.” The average score (Cronbach’s Alpha .88) was used as 
the participants’ self-efficacy score. We also measured the participants’ satisfaction with the instructional 
method with five items of which they had to indicate on a scale from 1 to 5 to what extent they agreed 
(1 = not at all, 5 = very much). The average score (Cronbach’s Alpha .76) was used as the participants’ 
satisfaction score. 

3. PRELIMINARY RESULTS AND CONCLUSION 

The score on the posttest has been evaluated with an analysis of covariance (ANCOVA) with Instructional 
Method (learning by doing), observational learning) and Reflection (yes, no) as the independent factors. Due 
to the small sample size, we only report mean scores here. First, we investigated the effect of instructional 
method and reflection on academic writing quality. The preliminary results indicate that there are no main 
effects for instructional method (M_observational learning = 8.17, M_learning by doing = 7.88) and 
reflection ( M_reflection = 8.08, M_no reflection = 7.89) on academic text quality. However, there seems to 
be an interaction between instructional method and reflection. In observational learning reflection appears to 
have a negative effect (M = 7.13) compared to no reflection (M = 9.00). In learning by doing the opposite 
seems to be the case. Reflection appears to have a positive effect (M = 8.56) compared to no reflection 
(M = 7.24). Secondly, we investigated the effect of instructional method and reflection on self-efficacy 
beliefs. The preliminary results indicate that there are no main effects of instructional method 
(M_observational learning = 71.86, M learning by doing = 70.07) and reflection (M_reflection = 70.05, 
M_no reflection = 71.36). Also, there seem to be no interactions. Thirdly, we looked at the satisfaction 
scores. The preliminary results suggest that participants who learn by doing (M = 3.94) are more satisfied 
with the instructional method than participants who learn by observation (M = 3.35). Reflections seems to 
have no effect on satisfaction (M_reflection = 3.75, M_no reflection = 3.73). Our preliminary results appear 
not to support our first two hypotheses. Observational learning does not lead to higher academic text quality 
or higher self-efficacy beliefs than learning by doing, and neither does reflection compared to no reflection. 
We have found some evidence for the third hypothesis. There appears to be an interaction between 
instructional method and reflection. In observational learning reflection seems to have a negative effect on 
text quality. This is not in line with Braaksma, et al. (2002). In learning by doing however reflection appears 
to have a positive effect on academic text quality. We hope to be able to draw more founded conclusion after 
we repeat the experiment. 
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